Sensitive Pictures:Emotional Interpretation in the Museum

Museums are interested in designing emotional visitor experiences to complement traditional interpretations. HCI is interested in the relationship between Affective Computing and Affective Interaction. We describe Sensitive Pictures, an emotional visitor experience co-created with the Munch art museum. Visitors choose emotions, locate associated paintings in the museum, experience an emotional story while viewing them, and self-report their response. A subsequent interview with a portrayal of the artist employs computer vision to estimate emotional responses from facial expressions. Visitors are given a souvenir postcard visualizing their emotional data. A study of 132 members of the public (39 interviewed) illuminates key themes: designing emotional provocations; capturing emotional responses; engaging visitors with their data; a tendency for them to align their views with the system's interpretation; and integrating these elements into emotional trajectories. We consider how Affective Computing can hold up a mirror to our emotions during Affective Interaction.Comment: Accepted for publication in CHI 202

Data-inspired co-design for museum and gallery visitor experiences

The capture and analysis of diverse data is widely recognized as being vital to the design of new products and services across the digital economy. We focus on its use to inspire the co-design of visitor experiences in museums as a distinctive case that reveals opportunities and challenges for the use of personal data. We present a portfolio of data-inspired visiting experiences that emerged from a 3-year Research Through Design process. These include the overlay of virtual models on physical exhibits, a smartphone app for creating personalized tours as gifts, visualizations of emotional responses to exhibits, and the data-driven use of ideation cards. We reflect across our portfolio to articulate the diverse ways in which data can inspire design through the use of ambiguity, visualization, and inter-personalization; how data inspire co-design through the process of co-ideation, co-creation, and co-interpretation; and how its use must negotiate the challenges of privacy, ownership, and transparency. By adopting a human perspective on data, we are able to chart out the complex and rich information that can inform design activities and contribute to datasets that can drive creativity support systems

ALTCAI: Enabling the Use of Embodied Conversational Agents to Deliver Informal Health Advice during Wizard of Oz Studies

We present ALTCAI, a Wizard of Oz Embodied Conversational Agent that has been developed to explore the use of interactive agents as an effective and engaging tool for delivering health and well-being advice to expectant and nursing mothers in Nigeria. This paper briefly describes the motivation and context for its creation, ALTCAI’s various components, and presents a discussion on its adaptability and potential uses in other contexts, as well as on potential future work on extending its functionality

Designing an adaptive embodied conversational agent for health literacy

ccess to healthcare advice is crucial to promote healthy societies. Many factors shape how access might be constrained, such as economic status, education or, as the COVID-19 pandemic has shown, remote consultations with health practitioners. Our work focuses on providing pre/post-natal advice to maternal women. A salient factor of our work concerns the design and deployment of embodied conversation agents (ECAs) which can sense the (health) literacy of users and adapt to scaffold user engagement in this setting. We present an account of a Wizard of Oz user study of 'ALTCAI', an ECA with three modes of interaction (i.e., adaptive speech and text, adaptive ECA, and non-adaptive ECA). We compare reported engagement with these modes from 44 maternal women who have differing levels of literacy. The study shows that a combination of embodiment and adaptivity scaffolds reported engagement, but matters of health-literacy and language introduce nuanced considerations for the design of ECAs